ALPR is an image-processing approach that often functions as the core module of “intelligent” transportation infrastructure applications. License plate recognition techniques, such as ALPR, can be employed to identify a vehicle by automatically reading a license plate utilizing image processing and character recognition technologies. A license plate recognition operation can be performed by locating a license plate in an image, segmenting the characters in the captured image of the plate, and performing an OCR (Optical Character Recognition) operation with respect to the characters identified.
The ALPR problem is often decomposed into a sequence of image processing operations: locating the sub-image containing the license plate (i.e., plate localization), extracting images of individual characters (i.e., segmentation), and performing optical character recognition (OCR) on these character images. In order for OCR to achieve high accuracy, it is necessary to obtain properly segmented characters.
The ability to extract license plate information from images and/or videos is fundamental to the many transportation business. Having an ALPR solution can provide significant improvements for the efficiency and throughput of a number of transportation related business processes.
A key component of any ALPR system is the ability to identify the state or jurisdiction of the license plate. Together with the plate code (string), the state uniquely identifies the license plate and its associated tag holder.
There are many challenges associated with accurately identifying the state of origin for the license plate. First, many vehicles these days have frames around the license plate that can obscure the written state name or motto. For example, FIG. 1 illustrates a prior art image 10 of a license plate in which the frame about the license plate obscures the state or jurisdictional name.
Other issues includes a lack of contrast and blur in captured images that can make it extremely difficult to recognize the state directly from the content outside the main plate characters. FIG. 2, for example, illustrates a prior art license plate image 12 having a blur and a lack of contrast. FIG. 3 illustrates sample prior art images 14, 16, 18 that compare fonts across three states for symbols “4” and “A” in accordance with an embodiment. FIG. 3 thus indicates that font issues also can present problems.
To combat these issues, the mainline technology in some ALPR systems leverages sequence information from the license plate string to determine the state of origin. In particular, the format of the character sequence (e.g., three letters followed by four numbers) along with the starting character in the sequence provides good performance for many applications. Since many states have their own unique formats and sequences, this type of an approach can provide accurate performance for many installations. A classifier is learned from training data that enables accurate prediction of the state based only on character sequence/format information. An example of this approach is disclosed in U.S. Patent Application Publication No. 2014/0348392 entitled “Method and System for Automatically Determining the Issuing State of a License Plate,” which published on Nov. 27, 2014 to Aaron Michael Burry, et al., and is incorporated herein by reference in its entirety.
There are several major limitations to this prior state of the art approach. First, as states increasingly issue more license plates, there is less separation between their valid sequences. For instance, the standard passenger car plate for both New York and Pennsylvania is three letters followed by four numbers (“LLL-NNNN”). The starting character for valid PA sequences begins with the letter “G” and up (e.g. “GAB-1234”, “HAB-1234”, etc.) whereas for NY the valid sequences start with the letter “A” (e.g. “AAB-1234”, “BAB-1234”, etc.). Unfortunately, as New York continues to issue new plates, they are in a sense “using up” the separation between the valid sequences for NY and PA. In other words, NY has started to issue plates that start with “G”, thereby creating confusion between NY and PA plates from the perspective of prior ALPR state identification methods.
Another limitation of the prior state of the art is that ALPR installations are looking to support recognition of more and more “out of jurisdiction” plates. Typically, for any given ALPR system the recognition engine is trained to support the main state and a handful, say five or six, surrounding states. Other plates have historically been considered out of jurisdiction and not eligible for automation. However, market forces are pushing towards supporting more and more states to enable higher and higher levels of automation. As we expand the set of supported states for any given installation, the opportunity for “overlap” between the formats and valid sequences grows. This is a stress on the accuracy of the existing state identification methods.
Part of the prior art disclosed in U.S. Patent Application Publication No. 2014/0348392 also involves leveraging information from the OCR step to assist in state identification. In particular, since the font can vary substantially from state to state, it is typical to train multiple OCR engines, each tuned to a particular state font. At runtime, all of the OCR engines are run in parallel and the one with the highest confidence result is selected as the “winner”. By incorporating this font selection information, the method of U.S. Patent Application Publication No. 2014/0348392 has demonstrated a benefit over just the mask plus starting character (MSC) technique alone. Unfortunately, the font selection information from a typical parallel OCR system alone is insufficient to provide the accuracy required by the ALPR market.